Distillation Sparsity Training Algorithm for Accelerating Convolutional Neural Networks in Embedded Systems

The rapid development of neural networks has come at the cost increased computational complexity. Neural are both computationally intensive and memory intensive; as such, minimal energy computing power satellites pose a challenge for automatic target recognition (ATR). Knowledge distillation (KD) can distill knowledge from cumbersome teacher network to lightweight student network, transferring essential information learned by network. Thus, concept KD be used improve accuracy networks. Even when learning there is still redundancy in Traditional fix structure before training, such that training does not situation. This paper proposes sparsity (DST) algorithm based on pruning address above limitations. We first through KD, then pruning, allowing learn which connections essential. DST allows teach pruned directly. proposed was tested CIFAR-100, MSTAR, FUSAR-Ship data sets, with 50% setting. First, new loss function teacher-pruned proposed, showed performance close Second, model (uniformity half-pruning UHP) designed solve problem unstructured facilitate implementation general-purpose hardware acceleration storage. Compared traditional UHP double speed